1. Field of the Invention
The present invention is suited for utilization in the medical field, and is related to a system and a method for assisting image diagnosis using three dimensional image data. The present invention is also related to a computer readable recording medium having a diagnosis assisting computer program stored thereon.
2. Description of the Related Art
In image diagnosis, there are cases in which images of a subject obtained during recent examinations (hereinafter, referred to as “current images”) are compared against images of the same subject obtained in previous examinations (hereinafter, referred to as “past images”), to confirm changes in symptoms of disease (hereinafter, referred to as “comparative image observation”). For this reason, many diagnosis assisting apparatuses are equipped with the function of selecting past images that represent the same position of a subject within an image which is currently being observed from a database of past images, and displaying the selected past images along with the observed current image.
A common user interface for comparative image observation displays a current image and a past image of the same size alongside each other on the screen of a monitor. An example of such a user interface is illustrated in FIG. 9, FIG. 11, and FIG. 12 of U.S. Patent Application Publication No. 20090080744. These figures illustrate screens of monitors, on which current images and past images having the same slice positions are displayed alongside each other during comparison of slice images obtained by a CT apparatus or the like. In addition, FIG. 15 of this document illustrates a case in which the contents of display of the monitor screen are switched between a current image and a past image by a scrolling operation. Further, a system in which two or more monitors are connected to a diagnosis assisting apparatus, a group of current images is displayed on one monitor, and groups of past images are displayed on one or a plurality of other monitors, is also proposed.
As another example of a user interface for comparative image observation is that in which only the regions of interest within two images are displayed close to each other (refer to FIG. 8, FIG. 9, FIG. 13, and FIG. 14 of Japanese Unexamined Patent Publication No. 2007-029248).
When using an interface that displays current images and past images alongside each other on the screen of a single monitor or on the screens of a plurality of monitors, physicians who observe the images must view the images while moving their lines of sight vertically and horizontally. Particularly in systems which are equipped with three or more monitors that are arranged vertically and horizontally, there are cases in which the physicians' heads must be rotated in addition to changing their lines of sight. These interfaces are likely to cause fatigue of the eyes and sore shoulders. On the other hand, when using an interface that switches between display of a current image and a past image by a scrolling operation, physicians must perform comparisons while one of the images is not within their fields of view (while retaining one of the images within their memories). This type of operation exerts a burden on the physicians' brains, and is likely to cause nervous fatigue.
Japanese Unexamined Patent Publication No. 2007-029248 proposes to reduce the amount of line of sight movement by displaying only the regions of interest close to each other. However, this document is silent regarding an image layout for cases in which a plurality of examinations have been performed in the past, and there is a great number of images to be compared.
The present invention has been developed in view of the foregoing circumstances. It is an object of the present invention to provide a user interface that reduces physical/mental fatigue of physicians that perform image observation.
A diagnosis assisting system of the present invention is a system equipped with a volume data storage means, a volume data selecting means, an image generating means, an image window control means, and a display control means, to be described below. In addition, a computer readable recording medium, on which a diagnosis assisting program of the present invention is recorded, causes one or a plurality of computers to function as the volume data selecting means, the image generating means, the image window control means, and the display control means, to be described below. The diagnosis assisting program of the present invention is constituted by a plurality of program modules. The functions of each of the aforementioned means are realized by one or a plurality of program modules. The group of program modules is provided to users by being recorded in storage media such as CD-ROM's and DVD's, by being recorded in a storage unit attached to a server computer in a downloadable state, or by being recorded in network storage in a downloadable state. A diagnosis assisting method of the present invention is a method that assists diagnosis, by causing a single computer or a plurality of computers to execute the processes of the volume data storage means, the volume data selecting means, the image generating means, the image window control means, and the display means, to be described later.
The volume data storage means stores a plurality of sets of volume data, obtained by imaging at least one subject on different imaging dates/times, in a predetermined storage device, correlated with subject identifying data and imaging date/time data. The storage device may be an internal memory or a storage of a computer that constitutes the diagnosis assisting system, an external storage device which is connected to the computer either directly or via a network, or the like.
Note that the term “subject” refers to a target of examination, that is, a portion which is the target of imaging and diagnosis. For example, in the case that the lungs and the stomach of a single patient are examined, there are two subjects, although there is only one patient. In this case, it is preferable for the subject identifying data to include both patient data and imaged portion data.
The volume data selecting means selects at least two sets of volume data correlated with subject identifying data that represents a specified subject, from among the sets of volume data stored in the storage device. Thereby, a plurality of sets of volume data regarding the same subject obtained on different imaging dates/times are obtained. For example, in the case that three CT examinations are performed on a subject, three sets of volume data, each of which is obtained at each examination, are obtained. Note that in cases that a plurality of sets of volume data are obtained by performing imaging using a plurality of modalities on a single imaging date, or by performing a plurality of imaging operations in temporal series during a single examination, there is a possibility that two or more sets of volume data are selected for a single imaging date.
The image generating means generates a plurality of images for observation in different observation formats that represent the specified subject for each set of volume data selected by the volume data selecting means, and generates a plurality of thumbnail images corresponding to each of the images for observation. The image generating means also correlates the images for observation, the thumbnail images, and the imaging date/time data of the set of volume data corresponding thereto. It is preferable for a positioning process to be performed such that the position of the subject within the images for observation and the thumbnail images generated from each set of volume data are matched among sets of volume data, when generating the images for observation and the thumbnail images. The positioning process may be administered onto the volume data, or onto the generated images for observation and the generated thumbnail images.
Note that the expression “different observation formats” refers to differences in the method of representation by which the subject is represented. That is, the portion of the subject which is focused on and how this portion is presented to a user differ. For example, images that are converted from volume data to two dimensional images by different conversion methods (the volume rendering method, the multi planar reconstruction method, etc.) are images having different observation formats. In addition, even if the conversion method is the same, images having different conversion parameters (viewpoint, direction of view, slice position, etc.) are images having different observation formats.
The image window control means arranges a plurality of image windows, which are switchable between an active state and an inactive state, on a screen, and controls the switching from the active state and the inactive state. Here, the active state refers to a state in which operational input can be received with respect to the contents displayed within an image window, and the inactive state refers to a state in which operational input cannot be received with respect to the contents displayed within an image window. The plurality of image windows may be windows that can be placed such that they overlap each other as necessary at a desired position on a display screen, or may be regions that the display screen is divided into. Further, it is preferable for the image window control means to be able to change the layout of the plurality of image windows in response to predetermined user operations.
The display control means selects images for observation which are correlated with imaging date/time data that represent a specified imaging date/time from among the images for observation generated by the image generating means, and displays the selected images for observation in the image windows. Here, the “specified date/time” needs only to specify a range of year/month/day/hour/minute/second which is sufficient to enable selection of images for observation. With regard to the selection of images for observation, in the case that a VR image and three MPR images corresponding to three cross sections are generated as the images for observation, the VR image and the three MPR images are each displayed within one of four image windows, for example.
Further, the display control means selects thumbnail images having the same observation format as that of the image for observation which is displayed in the active image window from among the thumbnail images generated by the image generating means, and displays the selected thumbnail images within the active image window. Having the same observation format refers to the subject being represented by the same method of representation. That is, having the same observation format refers to the aforementioned conversion method and the conversion parameters of the images being the same.
In the aforementioned structures and procedures, images which are targets for comparison do not appear on the screen, unless the image window that the image is displayed within is switched to the active state. For this reason, comparative image observation can be performed without confusion, even when a plurality of images for observation having different observation formats are displayed on the screen of a monitor. In addition, because the images to be compared are displayed as thumbnail images within a single image window, it becomes possible to compare the images without moving lines of sight greatly, even in cases that there is a great number if images to be compared. In addition, the need to prepare a plurality of monitors to perform comparative image observation is obviated.
However, during actual diagnosis, there are cases in which it is desired to comparatively observe only images obtained during a specific time period. For example, assuming that all of the results of periodic physical examinations are stored, there are cases in which it is desired to exclude images prior to an abnormality appearing and to use only images obtained after the abnormality was discovered, when performing comparative image observation. Alternatively, in the case that the number of images of a single subject is inordinately great due to series of examinations being performed over a long period of time or at high frequency, or due to images being obtained at a plurality of points in time accompanying ingestion of imaging agents, it is practically impossible to perform comparative image observation of all of the images. In such cases, it may be desired to narrow down the images when performing comparative image observation. The present invention proposes the following two means in response to such needs.
A first means is to configure the volume data selecting means such that it selects volume data that satisfies first narrowing conditions, to narrow sets of volume data which are targets for selection. Thereby, sets of volume data which are correlated with subject identifying data of a specified subject but do not satisfy the first narrowing conditions are not selected. Images for observation and thumbnail images are not generated from the sets of volume data which are not selected, and the display control means is enabled to display only images that comparative image observation is desired to be performed on. In addition, because only images that comparative image observation is desired to be performed on are generated, the processing load on the system is reduced, and processing efficiency is improved.
More specifically, the first narrowing conditions may be those that narrow the volume data to be selected to volume data which were obtained by imaging within a predetermined period. In this case, the volume data selecting means selects sets of volume data that satisfy the first narrowing conditions, based on the imaging date/time data correlated with the volume data. Alternatively, the volume data stored in the storage device may be further correlated with examination identifying data that identify each examination by which volume data were obtained, and the first narrowing conditions may be those that narrow the volume data to be selected to volume data which were obtained by a predetermined number of recent examinations. In this case, the volume data selecting means selects sets of volume data that satisfy the first narrowing conditions, based on the examination identifying data correlated with the volume data. As a further alternative, the volume data stored in the storage device may be further correlated with one of examination data that represent the examination during obtainment of the volume data and diagnostic result data that represent the results of diagnosis, and the first narrowing conditions may be those that narrow volume data to be selected to volume data which are correlated with one of a specific examination and a specific diagnostic result. In this case, the volume data selecting means selects sets of volume data that satisfy the first narrowing conditions, based on the examination data and the diagnostic result data correlated with the volume data. Here, the “specific examination or specific diagnostic result” refers to an examination results that indicate an abnormality, or a diagnosis indicating a specific disease.
A second means is to configure the display control means such that it selects thumbnail images that further satisfy second narrowing conditions, and causes the selected thumbnail images to be displayed within an image window in the active state. Thereby, thumbnail images having the same observation format as an image for observation which is displayed within an active image window but do not satisfy the second narrowing conditions are not selected as display targets. Therefore, only images that comparative image observation is desired to be performed on can be displayed. In addition, in this second means, the images for observation and the thumbnail images are generated even for those that do not satisfy the second narrowing conditions. Therefore, if the second narrowing conditions are changed or lifted, images which had previously not been targets for comparative image observation will be immediately available for display.
More specifically, the second narrowing conditions may be those that narrow the thumbnail images to be selected to thumbnail images which were obtained by imaging within a predetermined period. In this case, the display control means selects thumbnail images that satisfy the second narrowing conditions, based on the imaging date/time data correlated with the thumbnail images. That is, during selection of thumbnail images by the display control means, thumbnail images having the same observation format as an image for observation which is displayed in an active image window and correlated to imaging date/time data that represents an imaging date/time within the specified period are selected. Alternatively, the volume data stored in the storage device may be further correlated with examination identifying data that identify each examination by which volume data were obtained, the image generating means may correlate the examination identifying data, which are correlated to the volume data, to the images for observation and the thumbnail images generated therefrom, and the second narrowing conditions may be those that narrow the thumbnail images to be selected to be those which are generated from volume data which were obtained by a predetermined number of recent examinations. In this case, the display control means selects and displays thumbnail images that satisfy the second narrowing conditions, based on the examination identifying data correlated with the thumbnail images. As a further alternative, the volume data stored in the storage device may be further correlated with one of examination data that represent the examination during obtainment of the volume data and diagnostic result data that represent the results of diagnosis, the image generating means may correlate the examination data and the diagnostic result data, which are correlated to the volume data, to the images for observation and the thumbnail images generated therefrom, and the second narrowing conditions may be those that narrow volume data to be selected to volume data which are correlated with one of a specific examination and a specific diagnostic result. In this case, the display control means selects and displays thumbnail images that satisfy the second narrowing conditions, based on the examination data and the diagnostic result data correlated with the thumbnail images.
Alternatively, the display control means may receive settings regarding a maximum value for the number of thumbnail images to be displayed within an image window in the active state, and display a number of thumbnail images less than or equal to the set maximum value, regardless of whether the thumbnail images are narrowed as described above. Thereby, the displayed thumbnail images becoming too small and difficult to view due to the number thereof increasing can be prevented. Further, the display control means may receive selection of thumbnail images to be displayed, and display only the selected thumbnail images, in the case that the number of selected thumbnail images exceeds the maximum value. Thereby, it becomes possible to display only thumbnail images that a user desires to view.
As a further alternative, the display control means may receive settings regarding whether each of the thumbnail images within an image window in the active state are to be displayed, and only display thumbnail images which are set to be displayed. Thereby, it becomes possible for users to cause selective display of only thumbnail images that they wish to view. Further, the display control means may display positions at which thumbnail images, which have been set to not be displayed, had been displayed in a recognizable manner. By adopting this configuration, users will be enabled to be aware of the presence of the thumbnail images that were set not to be displayed, thereby preventing necessary images from being overlooked.
A configuration may be adopted, wherein the display control means displays a catalog of images for observation that correspond to thumbnail images which are displayed within an image window in the active state, in response to a predetermined operation by a user. By adopting this configuration, detailed comparative image observation can be performed using images for observation of large sizes, even in cases that the sizes of individual thumbnail images are too small and cause comparative image observation to become difficult, by the catalog of corresponding images for observation being displayed.
It is common for images to be observed while switching observation formats on a diagnosis screen, in image diagnosis employing volume data. Accordingly, it is preferable for the image generating means to generate an image for observation and a thumbnail image corresponding to a specified observation format, when an operation that requests a change in the observation format of an image for observation displayed in an active image window, and for the display control means to update the display to display the generated image for observation and the generated thumbnail image. It is particularly preferable for the updating of the image for observation and the thumbnail image to be performed synchronously, such that the observation formats of the images to be compared can be switched in a coordinated manner as a whole.
Note that a preferred specific example of an operation that requests a change in the observation format is an operation that specifies a region to be cut out within an image for observation which is displayed within an active image window. In this case, the image generating means generates a thumbnail image corresponding to the region which is to be cut out. Then, the display control means displays the region to be cut out within the image for observation in a recognizable format, and updates the display to display the generated thumbnail image.
The thumbnail images may be displayed within image windows in the active state such that they are overlapped on the edges of an image for observation, which had been being displayed also during the inactive state. Alternatively, an image for observation, which had been being displayed within an image window in the inactive state, may be reduced in size, and the thumbnail images may be arranged within the space created by the size reduction. However, there is a possibility that the image for observation becomes difficult to observe in an image window in the active state than in an image window in the inactive state, if these layouts are applied. Therefore, it is preferable for the display to be controlled such that the thumbnail images are arranged about the periphery of an image for observation, which had been being displayed from during the inactive state, when the image window in which the image for observation is displayed is switched to the active state.
For example, a configuration may be adopted, wherein:
the image window control means arranges a plurality of image windows having one constant display region and at least one temporary display region on the screen, and controls each image window such that the temporary display regions appear on the screen only when the image window it belongs in is in an active state. The constant display region is a region which is displayed regardless of whether the image window that it belongs to is in the active state or the inactive state. The temporary display regions are regions which are temporarily displayed only when the image window that it belongs to is in the active state. It is preferable for the temporary display regions to be set based on the number of selected thumbnail images or the like. It is also preferable for the temporary display regions to be of sizes such that they are within the field of view of a user with the constant display region.
By configuring the image windows as described above, displaying the selected images for observation within the constant display regions of the image windows, and displaying the selected thumbnail images in the temporary display regions of an image window in the active state, it appears to a user that an image window expands and the thumbnail images are displayed in the vicinity of the image for observation, when the image window is switched to the active state.
Note that it is preferable for the display control means to display an image for observation within the constant display region of an active image window corresponding to a specified thumbnail image, when a first operation that specifies a thumbnail image within a temporary display region is detected. By adopting this configuration, a user can sequentially perform the first operation with respect to a plurality of thumbnail images which are displayed in the temporary display regions, and observe each image in detail.
In addition, it is preferable for the display control means to replace a specified thumbnail image within a temporary display region with an image for observation corresponding to the specified thumbnail image, when a second operation that specifies a thumbnail image within a temporary display region is detected. By adopting this configuration, the image which is displayed within the constant display region and the image specified by a user are displayed as images for observation of the same size, and comparison of the two images is facilitated.
Further, it is preferable for the display control means to add and display the photography date represented by the photography date/time data correlated to each of the images for observation and/or the thumbnail images when displaying the images for observation and/or the thumbnail images. By adopting this configuration, confirmation of the imaging date/time of images during comparative image observation is facilitated.
According to the diagnosis assisting system, the diagnosis assisting method, and the computer readable recording medium on which the diagnosis assisting program of the present invention is recorded, a plurality of types of images are displayed on a diagnosis screen. Even in the case that a plurality of examinations has been performed in the past, and the number of images to be comparatively observed is great, users can perform comparative image observation with little fatigue.